Automatic feedback control is used in various fields such as: autopilots for aircraft, ships, and ground vehicles; industrial process control; factory automation; robotics; and other applications. In the context of this disclosure, “offline” means the controller parameters are pre-computed and stored. In contrast, “online” means the controller learns, and parameters are computed, as the system operates, e.g. as the aircraft flies. Computing and updating controller parameters using online solutions may allow for changing dynamics, for example, to handle the reduced weight of the aircraft as the fuel burns.
Conventional optimal feedback control design is performed offline by solving optimal design matrix equations. Furthermore, it is difficult to perform optimal feedback control designs for nonlinear systems since they rely on solutions to complicated Hamilton-Jacobi (HJ) or Hamilton-Jacobi-Isaacs (HJI) equations. A complete system dynamics model is needed to solve HJI equations, but such complete models are often difficult to obtain. Also, offline solutions do not allow performance objectives to be modified as the controller learns.